Two‐level preconditioning for Ridge Regression

Solving linear systems is often the computational bottleneck in real-life problems. Iterative solvers are only option due to complexity of direct algorithms or because system matrix not explicitly known. Here, we develop a two-level preconditioner for regularized least squares involving feature data matrix. Variants this may appear machine learning applications, such as ridge regression, logistic support vector machines and Bayesian regression. We use clustering create coarser level that preserves principal components covariance Gram This approximates dominant eigenvectors used build subspace accelerating Conjugate Gradient method. observed speed-ups artificial data.